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Executive Summary 

Project Goals 

The unique characteristics of migrant and seasonal 
families are important concerns for the Office of 
Head Start, yet the research and evaluation efforts 
that address this segment of the Head Start program 
have been limited. This study was designed to 
ascertain the state of research knowledge about 
Migrant and Seasonal Head Start (MSHS) programs; 
assess the feasibility of a range of measures and 
research methods; and select and pilot test methods, 
instruments, and procedures appropriate for this 
population. The Migrant and Seasonal Head Start 
Research Development Design Project was the result of 
the collaboration between MSHS programs and the 
Administration for Children and Families (ACF) to 
determine viable methods for assessing the unique 
characteristics of programs while also examining 
common programmatic components. The goals of 
the study were as follows: 

1. To "try out" methods and measures of key 
constructs for measuring both program 
implementation and child and family outcomes 
to determine whether methods and measures 
currently used in mainstream Head Start 
programs are feasible, and to identify potential 
new measurement approaches that may be 
warranted, particularly those focusing on child 
multilingualism and family environment; 

2. To identify the feasibility of tracking a sample 
of migrant families across sites, determining 
where they went, the programs in which they 
participated, and the degree to which continuity 
of services occurred within and between streams; 
and 

3. To develop methods to describe the issues 
and challenges faced by migrant and seasonal 
farmworker families and how MSHS programs 
operate to serve their unique needs. 

This report is meant to provide an overview of the 
issues in conducting research in these programs. 
Caution should be used in interpreting any specific 
results from this project because they are limited 
to the programs that participated in this project 
and thus should not be taken as being 
representative of all MSHS program; 


therefore, very few specific details regarding 
responses to measures are presented in this executive 
summary. The information that is provided should 
improve future research efforts attempting to 
accurately describe programs, families, and children 
of the MSHS community. 

Background 

The Indian and Migrant Programs Division was 
established in 1969, and in 1984 this division was 
further subdivided into two branches: the Migrant 
Programs Branch and the American Indian Programs 
Branch. Amendments to the Head Start Act, P.L. 105- 
285, sections 640(a)(2)(A) and 640(1)(2) established 
the eligibility of seasonal farm workers to receive 
services through the Migrant Programs Branch, and 
the name of the program was changed to Migrant 
and Seasonal Head Start. There have been few large- 
scale efforts to study MSHS families and programs. 

It would be difficult to accurately assess these 
programs within the methodological frameworks 
used in evaluation of Head Start mainstream 
programs (e.g.. Head Start Impact Study, Family 
and Child Experiences Survey [FACES]). There are 
several challenges to studying MSHS programs: they 
are organized differently and with more variations 
than mainstream programs. In addition, their 
program participants primarily speak Spanish and 
are highly mobile. 

MSHS shares the child and family goals of all Head 
Start programs. They strive to enhance children's 
growth and development; strengthen families as 
the primary nurturers of their children; provide 
high quality educational, health, and nutritional 
services; link children and families to needed 
community services; and have well-managed 
programs that involve parents in decision-making. 
MSHS endeavors to ready low-income children for 
successful transitions to elementary school. 

However, MSHS programs are flexibly structured 
to serve their diverse families. Not only are 
programs designed around the work schedules and 
migratory patterns of agricultural workers, but the 
services provided are also tailored to the particular 
competencies and needs of their families. Potentially, 
this results in an increased focus on child health, 
family employment and educational support, and 
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the parent-MSHS partnership. MSHS serves as a 
community resource for migrant families and can be 
of assistance as the families adapt to communities in 
different parts of the United States. 

Designing a comprehensive evaluation of MSHS 
programs, with a focus on child and family outcomes 
and program implementation, requires an initial 
concerted effort to begin to identify potential 
research methods and measures. The MSHS 
Research Design Development Project reviewed 
and piloted selected measures (i.e., interviews, 
questionnaires, direct assessments) with appropriate 
populations (i.e., staff, program directors, parents, 
children), assessed process features and feedback, 
and then further adapted the measures. Assessments 
of the process focused on the interaction between 
research and the cultural, theoretical, logistical, 
and demographic issues pertaining to migrant and 
seasonal farmworker families. Overall, the Design 
Development Project represents one step towards 
identifying useful evaluation methods and measures 
for future consistent study of these programs. 

Preliminary Identification of the 
Challenges of Researching MSHS 
Families 

The characteristics of MSHS programs produce 
significant practical problems for implementing 
research initiatives that will regularly inform the 
Office of Head Start and the MSHS community 
about program strengths and needs. This section 
presents the issues that emerged as a result of 
preliminary information-gathering activities that 
began when the project started. The information 
presented here is based on reading the literature 
regarding migrant and seasonal farmworkers; 
analyses of available datasets; and conversations 
and discussions with research experts, program 
providers, and policymakers. 

Language. Simply put, more than 95% of the MSHS 
families speak Spanish. In order for programs to 
serve as effective cultural and community resources 
for the families, MSHS makes a strong effort to hire 
staff and teachers with similar cultural and language 
backgrounds. Sometimes this effort is strengthened 
through programs involving parents themselves as 
staff or teachers. 


MSHS age groups. MSHS programs serve children 
from birth to five years of age; approximately half 
of the children served are under the age of three. 
Researchers considering evaluation of the 
programs should therefore vary their measures of 
program structure and teacher skills, depending 
on the age of the child in question. Further, direct 
assessment of these young at-risk children must be 
done with developmentally-appropriate measures 
and careful interpretations. 

MSHS eligibility variations. In order to qualify 
for the MSHS program, all MSHS families must 
earn their income primarily from agricultural work 
that involves the production and harvest of tree and 
field crops. The Head Start Act further mandates 
that "migrant" families participating in MSHS must 
have changed their residence from one geographical 
area to another within the last two years; "seasonal" 
farmworking families have not changed their place 
of residence from one geographical area to another in 
the last two years. In addition, the client population 
must meet the Head Start annual poverty 
guidelines. There are variations in programs' 
application of MSHS eligibility requirements (e.g., 
serving only migrant workers) which should 
influence the approaches used for assessing the 
programs as a whole. 

U.S. born versus immigrant families. Previous 
studies indicated that MSHS participants consist 
not only of migrant and seasonal farmworkers, 
but also of at least two distinct subpopulations 
of migrants. There are U.S.-born migrants from 
communities with a long history of migration and 
there are also recently arrived immigrant families 
who are seeking employment opportunities (Aguirre 
International, 1997). In 1997, two thirds of MSHS 
parents were identified as being born in the United 
States, while one third were born in other countries 
(Aguirre International, 1997). Further, the National 
Agricultural Workers Survey (2001-2002) reports 
that the proportion of migrant workers who were 
newly arrived in the United States increased by 69% 
between 1997 and 2002. Measurement and methods 
of a MSHS program survey should consider the 
possible language, culture, and service variations of 
these subgroups of migration families. 
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Geographic mobility. Since the late 19th century, 
migrant farmworkers have followed "streams" of 
changes in locality, depending on the harvest and 
production seasons. Downstream locations are those 
that have harvest and production seasons in the 
spring-summer seasons, while upstream locations 
tend to have harvest and production seasons in the 
late summer to late fall. According to tradition, there 
are three migrant streams, from south to north in 
the United States — Eastern, Midwest, and Western. 
In recent decades, however, these streams seem to 
have become less organized and distinct, perhaps 
reflecting inaccuracy in the early streamlined 
interpretation, or an increasing tendency for workers 
to either pursue cross-stream migrations or to stay 
in one location (i.e., settle out) while shuttling to 
various jobs relatively close to home. Some families 
also periodically return to their countries of origin 
during the non-growing seasons. This range of 
mobility indicates that accurate evaluations of the 
MSHS programs will have additional challenges if 
attempting to track families and measure long-term 
outcomes. Those challenges could include tracking 
within stream, across stream, across states, across 
programs, and even across countries. 

Continuity of MSHS services. Continuity of 
services refers to the sharing of information from 
one MSHS center to another regarding a migrating 
family. This can ease transitions and increase the 
supportive relationship between MSHS and the 
participating families. Theoretically, continuity 
efforts can occur within one program (if families 
change centers within a program) or between 
programs across the nation. However, there is 
currently no consistent national continuity effort 
across MSHS programs. Regional differences in the 
migratory patterns of MSHS families, as well as 
the degree to which different Head Start program 
grantees are organized, appear to influence the 
continuity of services. Programs with greater 
continuity structure (e.g., shared databases), more 
within-program mobility (e.g., family moves from 
one center to another within a program), or with 
more "settled out" families, will have more success 
with continuity efforts. For researchers, continuity 
is of particular interest if the families are going to 
be tracked over time in order to consider long-term 
MSHS outcomes and migratory patterns. 


This is only a brief overview of some of the 
characteristics of MSHS and migrant and seasonal 
families that would shape studies examining the 
programs. Numerous other features might also 
need to be considered in future research. Of high 
importance are the cultural considerations that 
could influence the families' perceptions of MSHS, 
education, and research. The assessment activities 
of the MSHS Research Design Development Project 
were designed to incorporate direct feedback 
from the families, staff, and teachers to increase 
the validity and utility of the research measures 
and methods. 

Design 

The project designers piloted and adapted 
interviews, direct assessments, and questionnaires 
and rating scales with the staff, teachers, children, 
and families of MSHS. The project consisted of a 
preliminary phase , a pilot study including focus groups, 
and on-site visits. The on-site visits usually involved 
a preliminary visit to introduce the project and on-site 
data collection visits , during which the full range of 
assessments and interviews were completed. 

Selection and development of measures continued 
throughout the project as interview formats and 
protocols were adjusted and adapted, incorporating 
the feedback from interviewers, parents, teachers, 
center directors, and staff. Of primary importance 
throughout the project was evaluation of the 
methods and measures themselves, in order to fully 
inform a future, larger-scale study of the MSHS 
program. This research development was one goal 
of the project. However, since the measures changed 
over time, any results describing the families and 
centers as a whole are necessarily limited. The 
evaluation included specific gathering of feedback 
through focus groups and debriefing questions, 
consideration of process and implementation 
features, and to the extent possible, careful 
analyses of the variability and reliability of the 
assessment tools. 

Preliminary Phase 

The preliminary phase included a literature review, 
discussions with key constituencies serving migrant 
and seasonal farmworker families, meetings with 
the Technical Work Group (TWG; see Appendix 
A), and focus groups with migrant families and 
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MSHS teachers and staff (MSHS Conferences, 2003 and 2004). These 
activities served to highlight the major issues to be addressed by the 
project, to shape the selection of measures and the development of new 
instruments, and to inform the field work procedures. 

Pilot Study 

A pilot study was designed to determine whether 1) the selected 
measures were understood by the respondents, 2) constructs could be 
translated for bilingual (Spanish-English) speakers, 3) scores on the 
children's assessments demonstrated levels of variation equivalent to 
those found with other Head Start groups, and 4) the data collection fit 
within reasonable time and burden limits. Four MSHS sites, selected on 
the basis of convenience, previewed the instruments, with approximately 
four parents and four teachers from each site participating in individual 
interviews and focus groups. Preschool-aged children of these 
participating parents completed pilot testing of the child assessments. 
These families and teachers were very cooperative and informative 
regarding their perceptions of the interviews and questionnaires, and 
their input resulted in several important adjustments to the measures 
and methodology. 

Site Visit Component 

The evaluators asked program sites to participate based on 
characteristics that were of particular relevance to the goals of the study: 
It is important to remember that this was not a representative sample of 
programs. For geographical and seasonal diversity, the project collected 
data from all three migrant streams — Eastern, Midwest, and Western — 
as well as data from both downstream and upstream sites. A total of 
seven sites participated in the on-site data collection (Florida, Texas, 
California, North Carolina, Tennessee (follow-up interviews only). North 
Dakota, and Oregon). See Figure 1 for a graphic representation of the 
study design. 

Preliminary visit. A preliminary visit to each center proved invaluable to 
increasing participation, staff support, and parent consent. This was an 
opportunity to share project goals and details, and answer participants' 
and staff's questions. 

Onsite data collection visits. For on-site data collection visits , researchers 
conducted child and family assessments; interviews with teachers, 
administrators, and program staff; and monitored the migration plans 
of families. Downstream data collection was done in April and July, 
while upstream data collection was done during September through 
November. Figure 2 indicates that the expectation was to collect 
data from 40 preschool-aged children per site for the preschool-aged 
child assessments and 40 families for the parent interviews. However, 
this plan assumed a one-to-one correspondence between families and 
preschoolers (i.e., one family=one child in the center), and did not take 
into account the fact that a large number of families had more than 
one child enrolled in the program. Given the sizes of the classrooms 
involved, it was not possible to select the requisite number of unrelated 
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preschoolers and infants/ toddlers and arrive at the target number 
of families. Only 134 preschool children were assessed. Since sibling 
participants appear to frequently occur in MSHS centers, it may work 
for future research projects to collect information about all participating 
family members. The unit of analyses in such a design could be at the 
family level (not child level) within MSHS centers, thus allowing the 
inclusion of siblings. 

Assessments and Interviews 

See Figure 2 for evaluations planned for each site. 

Child assessments. The direct child assessment for preschoolers was 
a battery of tests similar to those used in FACES, measuring different 
aspects of emergent literacy, language, and pre-academic skills. 1 The 
direct assessment for infants and toddlers consisted of the Preschool 
Language Scale Fourth Edition (PLS:4; Zimmerman, Steiner & 

Pond, 2002). 
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Teachers of preschoolers rated the children primarily in terms of social 
and classroom behavior (Teacher Child Report), whereas teachers of 
infants and toddlers were asked to rate the children across a number 
of developmental domains, using items from several well-known 
developmental checklists (Ages and Stages Questionnaire; Squires, 
Bricker & Potter, 1997; MacArthur Communication Developmental 
Inventory; Ariaga, Fenson et al., 1998; Minnesota Developmental 
Inventory; Saylor & Brandt, 1986). Parents also completed questionnaires 
considering the early development of their children. 


members. 


Language screeners 
were used as initial 
assessment tools. 


Dual language development (and, in some cases, multilingual language 
development) presents specific challenges for both children and 
program evaluation researchers. In order to identify preschoolers 
who were developing multiple language skills, language screeners 
were used as initial assessment tools. All preschoolers completed the 
Spanish language version of the screener tool and, if they passed, went 
on to complete additional assessments given in Spanish. The children 
were then given the English language screener and those who passed 
completed additional assessments given in English. 

Parent interviews. Individual in-person interviews were conducted 
with the parents of the study children. Topics included family 
background, demographics (socioeconomic circumstances, household 
structure, languages spoken in the home), parent-child interactions, use 
of social services, experience with MSHS programs, detailed questions 
about the parents' past work and migration patterns, use of community 
services (within the past year), and their future work and migration 
plans (within the 6 month period following the interview). The final 
section of the initial parent interview consisted of a formal debriefing 


1 For Spanish: Screener Spanish Pre-LAS, Test de Vocabulario en Imagenes Peabody, Spanish Letter-Naming, 
Spanish Woodcock-Johnson Letter-Word Identification, Spanish "Australian" Number Knowledge Test, Spanish 
Story and Print Concepts; For English: Screener English Pre-LAS, English Peabody Picture Vocabulary Test, 
English Letter-Naming, English Woodcock-Johnson Letter- Word Identification, English "Australian" Number 
Knowledge Test, English Story and Print Concepts Test 
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that asked parents to provide their reactions to the overall interview and 
to particular items. 

As noted in the introduction, development of the measures was a 
primary goal of the project. Perhaps more than the other measures 
in the Research Design Development Project, the parent interview 
changed frequently during the project. Adjustments were based on 
parent feedback and interviewer observations. Over the course of the 
preliminary phase pilot tests, the interview was shortened substantially 
from an average of 1.75 hours at the first site to one hour at the final site. 
(Refer to Interviews from the Migrant and Seasonal Head Start Design 
Development Project, Spanish Version and English Version). 

Center director and coordinator interview protocols. This 
component focused on how MSHS programs organized to offer useful 
support to migrant and seasonal farmworker families and their children. 
At six MSHS programs, a series of structured interviews were conducted 
with center directors/ coordinators/ staff and program directors. 

For simplification and accuracy, the original protocol changed from 
conducting two interviews for each of the MSHS centers (one for the 
program director and a second center staff survey), to a combined 
protocol that obtained information from the appropriate members of 
each given Head Start program community (e.g., education coordinator, 
health coordinator, family service workers, program and center 
directors). (Refer to Interviews from the Migrant and Seasonal Head 
Start Design Development Project, Spanish Version and 
English Version). 

Teacher interviews. The 19 teacher interviews were approximately 
30 minutes long and focused on classroom activities, curriculum, and 
teacher background and experience. During the preliminary phase, 
minor modifications were made to the teacher interview to eliminate 
redundancy and clarify concepts. The teacher interviews concluded 
with a 5-minute debriefing, asking the teachers to report regarding 
their interview experience. (Refer to Interviews from the Migrant and 
Seasonal Head Start Design Development Project, Spanish Version and 
English Version). 

Classroom observations. Additionally, structured classroom 
observations were completed in one preschool classroom in each center 
visited, using standardized observational methods widely used in child 
development research (Early Childhood Environment Rating Scale-R; 
Harms, Clifford, & Cryer, 1998; Arnett Scale of Caregiver Behavior; 
Arnett, 1989). 

Tracking and Locating Component 

A subcomponent of the study assessed methods of finding families 
once they had moved from the program in which they were originally 
assessed. This was to inform future research efforts regarding the 
feasibility of longitudinal assessment of program outcomes. Eighty 
families from the downstream Midwest (Texas) and Eastern (Florida) 
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streams formed the " tracking group" of study participants. Researchers 
identified the location of these families at two time points over a gap 
of 4 to 6 months to ascertain whether migrants could be found as they 
followed the harvest upstream. 

The "tracking" group of families was divided into two subgroups for 
consideration of two tracking methods. The multi-source method (N=18) 
for tracking families involved contact with program staff from both 
the sending and receiving MSHS programs and telephone contact 
with family members and the parents. The 18 families tracked with the 
multi-source methods were selected because of their stated migration 
plans: these families planned to migrate to 2 geographically clustered 
locations which simplified follow-up efforts. 2 The single-source method 
(N=61) 3 of tracking families involved only contact with program staff 
from both the sending and receiving MSHS programs. 


Figure 1. Migrant Streams and Planned Data Collection Timetable 



Learning From the Research Process: 
Obstacles and Adaptations 

The preceding section listed the measurements used for various 
members of the MSHS community (i.e., parents, children, teachers, staff). 
Beyond the initial implementation of the measures, a second important 


2 Eight of these families reported that they would be in Tennessee within the next six months, while 10 reported 
that they would be in South Dakota. 

3 One family withdrew from the program just after giving consent, and thus was not tracked. 



Time Period: April-June, 2004 1 September-November, 2004 


Figure 2. Summary of Original Project Plan, by Geographic Migration Stream and Time Period 
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• 35 Parent Interviews 


• 20 Retrospective Parent Interviews 


• 20 Parent Interviews 

• 15 Preschool- Age Child Assessments & 


• 1 Center Director Interview 


• 15 Preschool- Age Child Assessments & 

Teacher Ratings 


• 1 Program Director Interview 


Teacher Ratings 

• 10 Infant-Toddler Direct Assessments 




• 5 Infant-Toddler Teacher Ratings 

• 10 Infant-Toddler Teacher Ratings 




• 1 Preschool- Age Classroom Observation 

• 1 Preschool- Age Classroom Observation 




• 2 Teacher Interviews 

• 4 Teacher Interviews 




• 1 Center Director Interview 

• 1 Center Director Interview 




• 1 Program Director Interview 

• 1 Program Director Interview 







Multi-source Tracked Families 

Parent Follow-Up Interviews with 
10 Families Midwestern Upstream (North Dakota) 
8 Families Eastern Upstream (Tennessee) 

Single-Source Tracked Families 

61 Families (Various Locations) 



Western Downstream 
(California) 

40 Families 

• 40 Parent Interviews 

• 40 Preschool- Age Child Assessments & 
Teacher Ratings 

• 15 Infant-Toddler Teacher Ratings 

• 5 Infant-Toddler Direct Assessments 

• 2 Preschool- Age Classroom Observations 

• 4 Teacher Interviews 

• 2 Center Director Interviews 

• 1 Program Director Interview 
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Midwestern Downstream 
(Texas) 

40 Families 

• 40 Parent Interviews 

• 40 Preschool- Age Child Assessments & 
Teacher Ratings 

• 15 Infant-Toddler Teacher Ratings 

• 2 Preschool- Age Classroom Observations 

• 4 Teacher Interviews 

• 2 Center Director Interviews 

• 1 Program Director Interview 
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Eastern Downstream 
(Florida) 

40 Families 

• 40 Parent Interviews 

• 40 Preschool-Age Child Assessments & 
Teacher Ratings 

• 15 Infant-Toddler Teacher Ratings 

• 2 Preschool- Age Classroom Observations 

• 4 Teacher Interviews 

• 2 Center Director Interviews 

• 1 Program Director Interview 

1 


WEST 


MIDWEST 


EAST 


00 


Migration Stream 


The time and energy 
contributed by the 
families and staff 
to this process 
significantly and 
positively influenced 
the research 
development 
procedure. 


component of this project was assessing and developing the measures in 
the context of the MSHS programs. Were the types of questions posed 
well-received by the parents and program personnel? How could the 
interview process be facilitated? In addition, the researchers, ACF, and 
the MSHS programs wanted to make sure that the questions resulted in 
important and meaningful information. 

What follows is a description of the themes and challenges perceived 
by the participants, derived from the debriefing portion of the 
interviews and from focus group feedback. Often, the process 
information obtained from the respondents was applied directly to the 
interview protocols, resulting in adjustments to the interview questions 
over time. The time and energy contributed by the families and staff to 
this process was extensive, and their input significantly and positively 
influenced the research development procedure. 

Consent Rates 

There were significant differences in consent rate across sites (range 
39% to 100%). Several factors could have contributed to this variation. 
These include: 

Age of children: Consent rates appeared to vary by age of children, as 
the consent rate was somewhat higher for infants and toddlers (70%) 
than for preschoolers (63%). 

Data collection timing: The first two sites had lower consent rates 
compared to the last three sites, perhaps a result of researcher experience 
or families' migratory plans. 

Families’ degree of contact with the center. It seemed that the more 
established the parent-center relationships, the higher the consent rate 
for participation in the research. 

Wording of the consent form: Some individual families were hesitant 
to consent due to the concerns regarding mandatory reporting of 
potential child abuse to authorities. Discussion with MSHS staff eased 
these concerns for some parents. 

Regional cultural differences: Potentially, regional and program 
differences in families' culture, acculturation, and immigration status 
may also influence consent rates. 

Table 1 shows the number of cases completed for each measure. 

Learning From Parent Interviews/Focus Groups 

The parent interviews successfully gathered descriptive information 
from 194 parents. The majority of the parent respondents were the 
target child's birth parents (97%), followed by a grandparent and/ 
or a godparent (3%). Mothers, in particular, were the most frequent 
respondent (66%), although a substantial number of interview sessions 
included family members in addition to the mother (33%). 
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Table 1. Summary of Data Collection by Source and Instrument 



Target 

Cases 

Cases 

Response 

(see Figure 2) 

Selected 

Completed 

Rate 

Preschoolers 





Direct Assessment 

150 

142 

134 

94% 

Teacher-Child Ratings 

150 

142 

137 

96% 

Infants- 1 oddlers 





Direct Assessment 

15 

15 

15 

100% 

Teacher Ratings 

60 

59 

58 

98% 

Parents 





Full Initial Interview 

195 

194 

194 

100% 

Second-Child Interview 

30-50 1 


30 


Follow-up Interview 

15-20 1 

18 

17 

94% 

Teacher Interview 

18 

19 

19 

100% 

Classroom Observation 

8 

8 

8 

100% 

Directors/ Coordinators 

9 

9 

9 

100% 

Interview 





Grantee Director Interview 6 

6 

6 

100% 


When developing the parent interview, researchers continually 
counterbalanced the need to gather comprehensive information with the 
need to limit the length of the interview. Adaptations to the interview 
and themes gathered from parent feedback and focus groups are 
discussed below. 

Attitude towards research. Parents' level of trust in the center seemed 
to be associated with parents' approval of the research interviews and 
assessments. As one parent mentioned, "whatever the center says, we 
will follow because we trust them." Thus, in some cases, the research 
team received a "transfer" of confianza (trust) from the MSHS center 
staff. However, a few parents suggested that the term for research in 
Spanish (investigaciones) has a negative connotation that may need to 
be clarified to MSHS parents in order to increase participation. 

Interview length. Although the interview was progressively 
shortened throughout the research project, some parents continued to 
state that the interview was too long. Several parents felt that the tone 
and content of the parent interview were too similar to a psychological 
exam or a school test, and at times they were hesitant to respond 
to personal questions for fear of giving 'incorrect' answers. As the 
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interviewers developed familiarity with the questions over the course 
of the project, they tended to increase the conversational nature of the 
interview process and thereby may have reduced the " testing" tone. 

Translation limitations. The parents reviewed the translation of the 
questions, and inaccurate or awkward translations were identified and 
corrected. Some questionnaires were particularly repetitive when 
considered in translation (e.g., depression scale). 

Likert-type response scales. One major challenge to the 
methodology did arise during the original interview process. The 
original interview included a number of sections that involved the 
use of a Likert-scale, in which the parent was asked to consider a set 
of declarative statements and select their response to the statement 
from a 5-point scale. Descriptive phrases for each point on the scale 
indicated a level of quality (e.g., "not at all like me") or frequency (e.g., 
"very often"). Such Likert-type responses were used for sets of items 
regarding parenting beliefs, depression, and locus of control. MSHS 
parents were often reluctant to limit their answers with Likert-type scale 
or categorical responses. Questions using these methods often elicited 
complex answers that were prefaced by the words "it depends." Some 
parents apparently responded with "it depends" because they wanted 
to provide more exact and contextualized information than would be 
gathered by the fixed-choice format. 

Two strategies were attempted to make the Likert-type scales more 
effective. The first strategy was to limit the number of response options 
(e.g., three-point instead of five-point), while the second approach was 
to convert the Likert-type items into open-ended or narrative questions. 
Reducing the number of response choices (from 5-points to 3-points) was 
somewhat helpful when combined with the standard practice of using 
reminder cards to represent the 3-point scale. The alternative strategy, 
leaving questions open-ended in a narrative manner (in which parents 
could respond as they wish) was problematic because, without the 
additional structure, some parents felt that there must be a "correct" 
response to give. However, throughout the interview, whenever parents 
were asked to think of additional responses to fixed-choice questions 
(i.e.. Thank you for choosing from our categories; Is there any other 
way that you might respond?), many gave an informative variety of 
alternative responses. 

Parenting beliefs. There were no culturally relevant or appropriately 
standardized measures of parenting attitudes available at the time this 
project was initiated; The Parenting Beliefs Scale (previously used in 
FACES) was used in an exploratory attempt to consider these constructs 
with the MSHS families. This scale asks parents to report their level of 
endorsement of a series of statements regarding parenting attitudes. 
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The families did appear to have some consistent difficulties with this 
questionnaire. First, statements for the Parenting Beliefs Scale used the 
Likert-response method, which was often problematic. Many parents 


wanted to be particularly informative regarding variations in their 
beliefs and attitudes, depending on the context. In addition, at least for 
this sample, some parents of very young children felt that one or more 
of the Parenting Beliefs (i.e, "I encourage my child to be independent") 
were not developmentally appropriate. 

It might be particularly useful to ask for specific feedback regarding any 
measures of beliefs and attitudes, as they will be strongly influenced by 
culture and specific feedback will aid in interpretation. 

Access to services. For this section, interviewers first asked parents to 
identify which services they were receiving, using a list of community 
services. For each service that they were not receiving, the parents 
were then asked to choose a reason to explain why. The list of potential 
reasons worked well for some types of community services, but not for 
others. In order to accurately assess the service gap (between families' 
need for services and actually receipt of services), it appears that the 
questions may need to be asked more directly. 

Activities done with child by household members. Interviewers 
asked parents about interactive home activities for the child; these 
questions regarding the parent's or household members' involvement in 
activities with the study child appeared to work well. Overall, parents 
reported an average of seven activities per week with an informative 
amount of variation in their responses. It could be very informative 
to collect additional information about which household members 
performed which activities in order to identify the host of socialization 
and developmental resources available to the MSHS children. 

Questions about the amount of time parents read to their child, either 
during 1) the past week or 2) in one sitting, produced inconsistent data. 
From the results, it appears that some parents read to their child for zero 
minutes, but also read to their child at least once a week. There may be 
cultural differences in the perceived importance of literacy activities, and 
it is possible that misinterpretation or social desirability bias might be 
raised by the repeated questions on reading activities. 

Discipline. Prior to the implementation of the interview, there had 
been questions from focus groups and the Technical Work Group about 
whether parents would feel comfortable reporting about the full range 
of discipline activities. Of the parents responding to the discipline 
questions, 85% reported sending the child to a corner and 58% reported 
spanking (among other responses), suggesting that parents were 
relatively comfortable reporting about all discipline types. This 
might be further explored with specific feedback questions. In 
addition, discussion regarding such sensitive issues reminds 
researchers of the importance of clearly and accurately explaining 
research rights (e.g., confidentiality, right to refuse to participate) and 
limitations to those rights (e.g., abuse reporting laws), particularly with 
newly immigrant families. 


Parents would feel 
comfortable reporting 
about the full range of 
discipline activities. 


Parents provided 
information about 
the child assessment 
methods. 
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Some measures did not 
appear to differentiate 
among children with 
lower level abilities 
(i.e., a ‘floor’ effect). 


MSHS satisfaction. A number of parents chose to not respond to some 
of the questions regarding satisfaction with MSHS services. Future work 
should pursue feedback and clarification, as accurate measure of parents' 
satisfaction and suggestions will only serve to improve MSHS services. 

Suggestions for child assessment. Parents also provided 
information about the child assessment methods. Parents indicated 
that, in order for the assessments to be successful: 

• the evaluator would need to speak the child's language, 

• the evaluator should be skilled at adjusting the conversation to the 
child's vocabulary level, 

• sessions with children should be done over several days, 

• care should be taken to maximize familiarity with the evaluator, and 

• teacher presence would make the process smoother. 

Learning From the Child Assessment Process 

A number of important specific logistical issues were identified 
regarding the child assessments. These issues were identified from the 
teacher, parent, and evaluator perceptions of the assessments of the 
MSHS children: 

• presence of a teacher and a parent facilitated the child assessment, 

• assessments were perceived as more valid if conducted after the child 
was settled in program (for at least three weeks), 

• simplification of some assessment instructions appeared to reduce the 
effect of language on performance, 

• development of standardized additional probes regarding 
instructions also appeared to increase the accuracy of the results, 

• the Australian numeracy assessment (using concrete items to count) 
appeared to be less language-dependent and was therefore adopted 
as the measure of numeracy, and 

• screening the preschool children for both English and Spanish 
language skills was very important for assessment of these bilingual 
children in order to accurately identify skills in both domains. 

The assessment process was an extended trial of methods and measures, 
testing the applicability and acceptability of the procedures and 
gathering preliminary information about reliability and validity with 
these MSHS children. In other words, the MSHS Research Design 
Development Project addressed the feasibility of the measures and 
not the abilities of the children. Extensive analyses considering the 
variability of the measures, sensitivity to development, and overall 
reliability, resulted in the condensed information presented below. 
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There were some measures that did not appear to differentiate among 
children with lower level abilities (i.e., a 'floor' effect). These included. 


Table 2. Assessments of Child Assessment Measures: Usefulness for 
Assessment of the MSHS Sample 


Spanish Tests English Tests 


Child Assessment 
Measure 

Standard 

Error 1 

Reliability 2 

Standard 

Error 1 

Reliability 1 

Language Screener 
(Pre-LAS) 

Excellent 

Excellent 

Excellent 

Excellent 

Vocabulary 

(TVIP/PPVT) 

Excellent 

Excellent 

Excellent 

Good 

Letter Naming 

Excellent 

Good 

Excellent 

Excellent 

Letter- Word 
Identification 
(Woodock- Munoz/ 
W oodcock-J ohnson) 

Marginal 

Poor 

Good 

Good 

Number 
Knowledge Test 

Marginal 

Marginal 

Excellent 

Good 

Story and Print 
Concepts 

Marginal 

Poor 

Marginal 

Marginal 


1. Excellent = SEM's < .37, Good = SEM's < .49, Marginal = SEM's < .59, and Poor = SEM's > .59. 

2. Excellent = reliability coefficients > .85, Good = coefficients > .75, Marginal = coefficients > .65, Poor = coefficients < .65. 


Forty percent 
of children were 
administered only 
the full Spanish 
assessment while 
27% of the children 
attempted only 
the full English 
assessment. 


for example, the Spanish Letter- Word Identification and the English and 
Spanish Story and Print Concepts tests. Future work with larger samples 
should be closely considered to further evaluate the viability of these 
measures with these populations. The measures did demonstrate an 
appropriate tendency to become easier across the age groups assessed 
(i.e., the older the child, the easier each of the tests tended to be). Table 
2 reflects interpretation of the Standard Error of Measurement (SEM) 
identified for each assessment tool; standard errors of measurement 
provide a measure of test precision. No measure should be rejected out 
of hand based on results from such a small sample size (N=134), but the 
interpretations suggest areas where there is a potential need for further 
study and alternative measures. 

Preschool assessment. One hundred and thirty four assessments 
were completed; only 7% of the preschoolers were either too shy or 
too restless to complete the procedures. Preschool children assessed 
for this project tended to be at the younger end of the preschool age 
range, as 41% of the sample was 3 years old. Each child attempted 
the Spanish and English language screeners in order to measure their 
abilities to try the additional assessments in each language. The largest 
proportion of the children (40%) were administered only the full Spanish 
assessment (which means they passed the Spanish screener but not the 
English screener) while 27% of the children passed enough items on the 


Twenty-six percent 
passed both 
screeners and thus 
attempted both the 
Spanish and English 
full assessments. 
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English screener but not the Spanish Screener, and 
therefore attempted only the full English assessment. 
One quarter of the children (26%) passed both 
screeners and thus attempted both the Spanish and 
English full assessments. These findings suggest, 
in general, that the separate language screener 
approach is potentially viable with this population. 

Infant/toddler assessment. With regards to 
the infant and toddler assessment (the Preschool 
Language Scale 4th Edition), this small exploratory 
portion of the study was only able to demonstrate 
that direct assessment of MSHS infants and toddlers 
was possible (N=15). Assessments with these young 
children occurred only when the parent was 
present in order to increase the child's comfort 
level. Some attending parents appeared unsure of 
their role in the assessment process, and further 
standardized coaching for parents might reduce 
parent and child anxiety and increase parental 
assistance during the assessment. 

Ratings. The teacher responses to ratings of 
preschoolers' social skills, early literacy skills, and 
engagement in learning appeared to result in an 
appropriate range of variability. 

For the infants and toddlers, their teachers 
completed developmental checklists. Although they 
may have some value when considered in a larger 
sample, many items on the developmental scale 
demonstrated either a ceiling or floor effect. In other 
words, teachers tended to think that some emergent 
literacy and language items would generally be 
too difficult for their classroom's children, while 
fine and gross motor items reflected activities that 
were too easy. Adjustments and refinements of 
the infant/ toddler developmental rating scales, or 
identification of alternative established scales, should 
be considered for future projects. 

Learning From the Teacher Interviews 

Separate interviews were developed for teachers 
of infant/ toddlers and teachers of preschoolers. 
About three fourths of the teacher interviews were 
conducted in Spanish (by teacher choice). Interviews 
were conducted with 19 teachers in 5 of the 6 MSHS 
project sites: 7 teachers in infant/ toddler rooms and 
12 teachers in preschool classrooms. Clearly, 
this was a very small and non-representative 


sample, and allowed for only very preliminary 
exploration of the range and variety of responses to 
the interview questions. 

Topics included classroom population, typical 
daily activities, classroom environmental features, 
developmental knowledge, languages and 
curriculum used, and administrative supports 
(such as mentoring and coaching). Teachers were 
also asked about the frequency and format of 
communications and interactions with parents. 
Finally, teachers were asked about their perceptions 
of the programs, satisfaction with their work, and 
their plans to return to MSHS the following year. 

In the focus groups, done as part of the pilot study 
for the project, teachers discussed the kinds of 
assessments used with children and what they felt 
were the most critical outcomes of MSHS programs. 
Teachers hoped to make differences in the lives of 
the families and they discussed numerous outcomes 
that could reflect the impact of MSHS, including: 

• parents' knowledge of the importance of a 
healthy child and family; 

• parents' attitudes towards education; 

• family awareness of community resources, 
particularly literacy and ESL supports; 

• parent awareness of adult educational 
opportunities; 

• father involvement; 

• parent participation in MSHS activities; 

• increased socialization; and 

• increased communication. 

The teachers urged the evaluators to "spend 
time in their classrooms," in order to observe the 
children within the educational settings. Teachers 
listed a number of changes in children's skills and 
development that could reflect the impact of MSHS 
programs. These included: 

• improved self-esteem, 

• improvements in motor skills, 

• earlier identification of developmental delays or 
special needs, 

• increased appropriate social play, 

• language development, and 

• school readiness. 
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These MSHS teachers generally felt that it was important that 
outsiders know that the MSHS programs make a difference for both 
parents and children. 

Suggestions for researchers. Although these teachers tended 
to be supportive of MSHS research, they emphasized the need for 
confidentiality and a sensitive understanding of the impoverished 
conditions of the community that they served. Teachers also cautioned 
evaluators not to make the mistake of "labeling" the MSHS children 
when using standard evaluative tools. 

Likert-scale. Teachers tended to find the Likert-scale responses to be too 
limited, much as did the MSHS parents: Likert-style questions regarding 
child development often produced an "it depends" answer. Teachers 
frequently wanted to emphasize the range of student behavior and not 
focus on the average. They noted that the frequency of behaviors would 
depend on the age, the amount of prior exposure to MSHS classrooms, 
and even the time of the year that the teacher is asked the question. 

Age and salary. Teachers were consistently very hesitant to give 
their age and salary, reportedly due to discomfort with sharing such 
information, and sometimes due to concern that the numbers would be 
used to assess their relative abilities and cost to the programs. 

Uniform questions. Some questions appeared to address issues 
that were either very uniform across programs or had a heightened 
sociability bias. 

Teachers reported very uniform answers for: 

• all three job satisfaction questions, 

• presence of educational toys in the classroom, 

• scheduling and conducting parent conferences /home visits, and 

• curricula in use in classroom. 

Questions that do not provide useful information regarding variation 
across programs should be dropped from future research. 


Teachers listed a 
number of changes 
in children’s skills 
and development 
that could reflect 
the impact of MSHS 
programs. 


Teachers tended to 
find the Likert-scale 
responses to be too 
limited. 


Center directors 
identified five parent- 
oriented goals as 
‘most important’ for 
MSHS programs. 


Learning From Center Director/Coordinator Interviews 

The final version of the Center Director/ Coordinator Interview was 
designed as one protocol, combining information from multiple 
sources in order to reduce redundancies and effort required from 
the program. Copies of the interviews were sent to the sites ahead of 
time so that center directors could complete a preliminary summary 
of some of the more "factual" data about their centers and programs, 
such as the number and ages of children, the number of children from 
migrant versus seasonal families, and the numbers and types of staff 
working at the center. Staff and the center director were asked to address 
the following issues in the direct interview: agreements with community 
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resources; enrollment, waiting lists, and program 
expansion efforts; health services and health 
concerns for client population; teacher training and 
role of the education coordinator; teacher turnover; 
Family Needs Assessments; assessments used to 
measure children's progress; kindergarten transition 
activities; parent involvement activities; and staff's 
perceptions of MSHS. 

Center directors identified five parent-oriented goals 
as 'most important' for MSHS programs: 

• informing parents about their own child's 
development, 

• helping parents identify their personal goals and 
ways to achieve them, 

• informing parents about the support services in 
their community and helping them to use them, 

• helping parents become economically self- 
sufficient, and 

• helping parents improve their literacy skills. 

When center directors were asked to report on two 
activities that the MSHS programs performed well 
for children and families, the following responses 
were given: 

• "Provide the services for children and for parents, 
for them to feel that we are not just here for a 
paycheck but they can come to us if they have a 
problem." 

• "We provide the best quality child care services to 
the children." 

• "We support the values of the family and value 
each as unique individuals." 

• Communication with and information given 
to parents through conferences, home visits, 
screening, and parent meetings; 

• Provide services in physical health and speech 
to parents and children and work with other 
agencies in making sure they get these services; 
and 

• Provide a safe and pleasant environment. 

These themes were generally reiterated by the health 
coordinators, education coordinators, and family 
service workers. 


When asked for areas for improvement, the most 
common issues raised were adjustment of MSHS 
qualification and the income eligibility requirement 
for families, as the guidelines limit the families that 
could be served. Additional issues mentioned were 
staff salaries, staff turnover, maintaining health and 
education coordinators as separate positions, and 
the length of times that programs were open each 
year. Leadership and administration training were 
also discussed. 

Overall, the interviews with the center directors 
and coordinators went well and respondents gave 
thoughtful answers, though a few procedural 
issues were observed. As with teachers, the center 
directors and staff were reluctant to discuss age 
and salary. There were again several questions that 
received consistently uniform responses across 
sites, suggesting that the questions will not provide 
differentially useful information (e.g., problems 
faced in expansion, parent participation efforts, 
parent communication efforts, etc.). This could 
partly be caused by social desirability bias (e.g., 
directors wanting to endorse a full set of positive 
activities for their center) and partly because the 
behaviors may actually be universally present in 
MSHS programs. Given a long interview, questions 
where little variation in answers was found should 
be either eliminated or adjusted to increase the 
amount of new information that can be obtained. 


Learning From the Tracking and Locating 
Component 

To measure long-term outcomes of program 
participation, families and children must be 
followed as they change locations so that data 
collection can take place at least two time points. The 
MSHS Research Design Development Project hoped 
to assess the extent to which the families sampled in 
downstream locations could be located at upstream 
MSHS program locations. As discussed above, 
the tracking sample for this project consisted of 80 
participating families from two downstream sites. 

The two upstream areas where migrant families 
in the project would be geographically clustered 
were identified during the initial interview, one in 
Tennessee and another in North Dakota. For these 
geographically clustered families (N=18), multi- 
source tracking methods were used (i.e., contact 
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with program staff, family members, and parents). Although one family 
withdrew from the study, 61 other families were tracked using a single- 
source method (i.e., contact with programs' sending and receiving 
staff). Parents from both groups were also asked to complete periodic 
postcards, notifying project staff of moves. Respondents were paid $5 for 
each returned postcard. 


Table 3. Tracking of Families from Downstream to Upstream 



Multi-Source 
(n = 18 ) 
Mean 

Single-Source 
(n = 61 ) 
Mean 

Total 
(N = 79) 

Mean 

Total No. of Attempts 
to Contact Families 

6 

2 

3 

Days Spent Contacting 

57 

16 

25 


N Percent 

N Percent 

N Percent 

Families Found 

17 

94% 

54 89% 

71 90% 

Winter Location Matched 
Parent Report in Spring 

14 

82% 

21 39% 

35 49% 

Study Children with 
Family in New Location 

26 

100% 

67 85% 

93 89% 


Tracking and locating success rate. Overall, the tracking and locating 
efforts were successful, albeit with some differences in methods and 
success rates for the multi- versus single-source samples (see Table 3). 

On average, the multi-source tracking sample involved more than twice 
as many contact attempts and took more than three times longer than 
the single-source methods. The success rate for the multi-source tracking 
sample was slightly higher than that for the single-source tracking 
sample (94% and 89% respectively), although the researchers suggest 
that the relatively small differences may not justify the significantly 
larger effort. Only seven families sent the postcards in over the course 
of the project, so this intervention did not appear to increase tracking 
effectiveness. 

It is important to note that this locating and tracking effort was to 
assess the possibility of finding families over time in order to allow for 
longitudinal analyses of MSHS outcomes and family migratory patterns 
in future research efforts. It was not planned to address the continuity 
possibilities between MSHS programs. 


The multi-source 
tracking sample 
involved more than 
twice as many 
contact attempts and 
took more than three 
times longer than 
the single-source 
methods. 
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Conclusion 

The invaluable feedback and cooperation offered 
by the MSHS programs, parents, and children 
allowed for the initial development of measures 
and methods appropriate for MSHS populations. 
This project could not have been completed without 
substantial input from the Technical Work Group 
as well as from a variety of administrators and 
participants of the MSHS programs. The researchers 
also gained additional perspectives from attending 
and presenting the project to the national MSHS 
conferences. 


Consistent and detailed descriptive assessments 
of MSHS programs, staff, parents, and children 
would provide invaluable information to Office 
of Head Start. Although this project's exploratory 
nature did not allow for any firm conclusions about 
the evaluation methods that should be applied to 
MSHS programs, multiple lessons were learned 
regarding methodology and implementation. These 
lessons were derived from development and direct 
implementation of measures with the appropriate 
populations; incorporation of the resulting feedback 
and observation will lead to even more effective 
and valid measures and methods in future MSHS 
studies. 
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Appendix A: Membership of the Technical Work Group 


Ms. Manda Lopez 

Migrant & Seasonal Head Start Association 
4350 North Fairfax Drive, Ste. 410 
Fairfax, VA 22203 

Ms. Carole Clarke - Program Director 
Pajaro Valley Unified School District 
294 Green Valley Road 
Watsonville, CA 95076 

Ms. Mary Ellen Good 

Director of Migrant Clinician's Network 

1736 Eastwood Court 

Fort Collins, CO 80525 

Mr. Rufino Dominguez-Santos 
Frente Indigena Oaxaqueno Binacional 
PO Box 106 
Fresno, CA 93707 

Mr. Rafael Guerra 

Project Director - East Coast Migrant 
Head Start Project 
4245 North Fairfax Drive, Ste. 800 
Arlington, VA 22203 

Dr. Spencer Salend 
SUNY at New Paltz 
Migrant Special Education Program 
Room 112 Old Main Building 
New Paltz, NY 12561 

Dr. Philip Martin 

Department of Agricultureal Economics 
1 Shields Avenue 
Davis, CA 95616 


Dr. Carol Hammer 

Penn State Dept, of Communication 

Sciences and Disorders 

0110 Moore Building 

University Park, PA 16802 

Dr. Barbara Pearson 

Department of Communication Disorders 
NIH Working Groups on AAE 
University of Massachusetts 
11 Paige Street 
Amherst, MA 01001 

Dr. Richard Mines 
California Institute for Rural Studies 
221 G Street, Ste. 204 
Davis, CA 95616 

Ms. Donna Marie Marlow 
Office of Migrant Education 
U.S. Department of Education 
400 Maryland Ave., SW 
Washington, DC 20202-6135 

Ms. Michelle Saucedo 

Attn: Alicia Santana, Parent Coordinator 

ECMHSP Project 

131 Third Street, SW, 2nd Floor 

Winter Haven, FL 33880 
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